对比语言 - 图像预培训（剪辑）在广泛的图像中与跨模仿监督学习的卓越成功 - 在线收集的文本对。到目前为止，夹子的有效性主要是在一般结构域多数制问题中进行研究。这项工作评估了剪辑的有效性，用于医学视觉问题的任务（MedVQA）。为此，我们向PubMedClip提供PubMedClip，基于PubMed文章的医疗领域的微调版本。我们的实验是在两个MedVQA基准数据集中进行，并调查两种MedVQA方法，MEVF（增强的视觉功能）和QCR（通过条件推理的问题回答）。对于这些中的每一个，我们使用PubMedClip，原始剪辑和最先进的MAML（模型 - 不可知的Meta-Learning）网络仅评估视觉表示学习的优点，仅在视觉数据上训练。我们为我们的Medvqa管道和预训练PubMedclip开源代码。与MAML的Visual Encoder相比，剪辑和PubMedClip实现了改进。 PubMedclip以最高精度的最佳效果达到最佳结果，高达3％。个别示例说明了与先前广泛使用的MAML网络相比的PubMedclip的强度。 PubMedclip语言监督的视觉表现出学习导致MedVQA的显着改进。我们的实验揭示了在以前的工作中尚未传授的两个MedVQA基准数据集中的分布差异，并在PubMedClip中导致不同的后端视觉编码，在这些数据集上表现出不同的行为。此外，我们证明了VQA一般与医学领域的基本性能差异。

深度神经网络用于图像识别任务（例如预测笑脸）的性能会以代表性不足的敏感属性类别降低。我们通过基于人口统计学奇偶校验，均衡赔率和新型的联合会措施的批估计估计来引入公平意识的正规化损失来解决这个问题。对Celeba，UTKFACE和SIIM-ISIC黑色素瘤分类挑战的面部和医学图像进行的实验表明，我们提出的公平性损失对偏置缓解的有效性，因为它们可以改善模型公平，同时保持高分类性能。据我们所知，我们的工作是首次尝试将这些类型的损失纳入端到端培训方案，以减轻视觉属性预测指标的偏见。我们的代码可在https://github.com/nish03/fvap上找到。

It is common practice in deep learning to represent a measurement of the world on a discrete grid, e.g. a 2D grid of pixels. However, the underlying signal represented by these measurements is often continuous, e.g. the scene depicted in an image. A powerful continuous alternative is then to represent these measurements using an implicit neural representation, a neural function trained to output the appropriate measurement value for any input spatial location. In this paper, we take this idea to its next level: what would it take to perform deep learning on these functions instead, treating them as data? In this context we refer to the data as functa, and propose a framework for deep learning on functa. This view presents a number of challenges around efficient conversion from data to functa, compact representation of functa, and effectively solving downstream tasks on functa. We outline a recipe to overcome these challenges and apply it to a wide range of data modalities including images, 3D shapes, neural radiance fields (NeRF) and data on manifolds. We demonstrate that this approach has various compelling properties across data modalities, in particular on the canonical tasks of generative modeling, data imputation, novel view synthesis and classification. Code: https://github.com/deepmind/functa

联合学习（FL）是一项广泛采用的分布式学习范例，在实践中，打算在利用所有参与者的整个数据集进行培训的同时保护用户的数据隐私。在FL中，多种型号在用户身上独立培训，集中聚合以在迭代过程中更新全局模型。虽然这种方法在保护隐私方面是优异的，但FL仍然遭受攻击或拜占庭故障等质量问题。最近的一些尝试已经解决了对FL的强大聚集技术的这种质量挑战。然而，最先进的（SOTA）强大的技术的有效性尚不清楚并缺乏全面的研究。因此，为了更好地了解这些SOTA流域的当前质量状态和挑战在存在攻击和故障的情况下，我们进行了大规模的实证研究，以研究SOTA FL的质量，从多个攻击角度，模拟故障（通过突变运算符）和聚合（防御）方法。特别是，我们对两个通用图像数据集和一个现实世界联邦医学图像数据集进行了研究。我们还系统地调查了攻击用户和独立和相同分布的（IID）因子，每个数据集的攻击/故障的分布对鲁棒性结果的影响。经过496个配置进行大规模分析后，我们发现每个用户的大多数突变者对最终模型具有可忽略不计的影响。此外，选择最强大的FL聚合器取决于攻击和数据集。最后，我们说明了可以实现几乎在所有攻击和配置上的任何单个聚合器以及具有简单集合模型的所有攻击和配置的常用解决方案的通用解决方案。

Human observers can learn to recognize new categories of images from a handful of examples, yet doing so with artificial ones remains an open challenge. We hypothesize that data-efficient recognition is enabled by representations which make the variability in natural signals more predictable. We therefore revisit and improve Contrastive Predictive Coding, an unsupervised objective for learning such representations. This new implementation produces features which support state-of-theart linear classification accuracy on the ImageNet dataset. When used as input for non-linear classification with deep neural networks, this representation allows us to use 2-5× less labels than classifiers trained directly on image pixels. Finally, this unsupervised representation substantially improves transfer learning to object detection on the PASCAL VOC dataset, surpassing fully supervised pre-trained ImageNet classifiers.

We present a framework for efficient inference in structured image models that explicitly reason about objects. We achieve this by performing probabilistic inference using a recurrent neural network that attends to scene elements and processes them one at a time. Crucially, the model itself learns to choose the appropriate number of inference steps. We use this scheme to learn to perform inference in partially specified 2D models (variable-sized variational auto-encoders) and fully specified 3D models (probabilistic renderers). We show that such models learn to identify multiple objects -counting, locating and classifying the elements of a scenewithout any supervision, e.g., decomposing 3D images with various numbers of objects in a single forward pass of a neural network at unprecedented speed. We further show that the networks produce accurate inferences when compared to supervised counterparts, and that their structure leads to improved generalization.